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Testing is a de-facto verification technique in industry, but insufficient for identifying subtle issues 
due to its optimistic incompleteness. On the other hand, model checking is a powerful technique that 
supports comprehensiveness, and is thus suitable for the verification of safety-critical systems. How- 
ever, it generally requires more knowledge and cost more than testing. This work attempts to take 
advantage of both techniques to achieve integrated and efficient verification of OSEK/VDX-based 
automotive operating systems. We propose property -based environment generation and model ex- 
traction techniques using static code analysis, which can be applied to both model checking and test- 
ing. The technique is automated and applied to an OSEK/VDX-based automotive operating system, 
Trampoline. Comparative experiments using random testing and model checking for the verification 
of assertions in the Trampoline kernel code show how our environment generation and abstraction 
approach can be utilized for efficient fault-detection. 



1 Introduction 

The operating system is the core part of automotive control software; any malfunction can cause critical 
errors in the automotive system, which in turn may result in loss of lives and assets. Testing has been 
widely used as a systematic and cost-effective safety analysis/assurance method |[6l[T8l, but its optimistic 
incompleteness often misses critical problems and cannot guarantee the "absence of wrong behavior". 
As an alternative and complimentary technique, model checking HU1 [16) has been drawing attention 
from both academia and industry. 

Model checking is a comprehensive formal verification technique, suitable for functional safety anal- 
ysis. It can effectively identify subtle issues, such as process dead lock, illegal behavior, and starvation, 
but may require more resources and domain knowledge. In particular, the use of model checking faces 
the following challenges: 

1. The size of model/code to be verified needs to be minimized to avoid state-space explosion. 

2. Modeling of the environment, such as user tasks and hardware environment, is necessary and 
critical for embedded software. 

Since an operating system is a reactive system responding to environmental stimuli, the correctness 
of its behavior needs to be analyzed with respect to the behavior of its environments. A non-deterministic 
environment is typically used to over-approximate actual behavior, but it is often too expensive in model 
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checking. The difficulty and importance of defining a good environment model has been addressed in a 
number of previous works E21 1251 1121 1251 [171 1201. 

We note that these two problems apply to both model checking and testing. Though the level of 
comprehensiveness differs, both techniques rely on automated search techniques that are initiated by en- 
vironmental stimuli. This is called environment model in model checking and test scenario in testing. 
This work anticipates that the efficiency of automated verification techniques depends on the modeling 
of the environment and proposes an application of property-based code slicing [24j for automatically 
generating an environment model using the data/function dependency analyzed from the operating sys- 
tem kernels. The goal is to construct a valid and comprehensive usage model of the operating system 
with minimal dependency on the kernel code. 

Our approach extracts functions that have a direct dependency on a given property to be verified 
and generates non-deterministic function-call sequences by imposing (1) external constraints from the 
OSEK/VDX standard O for automotive operating systems, and (2) internal constraints identified from 
the function call structure of the operating system kernel. The external constraints are manually identified 
from the specifications of the standard and are imposed on the initially random sequence of function calls. 
The internal constraints are imposed by identifying the top-level functions using backward slicing from 
a given property and by computing the cone-of-influence from each top-level function using forward 
slicing. The Environment model is defined as an arbitrary sequence of calls of those extracted functions. 
The operating system kernel is also abstracted as a collection of extracted functions and its relevant code 
required for pre-processing them. This procedure reduces the size of the verification target and minimizes 
the behavior of the environment model. The extraction and model construction process is automated with 
the aid of the static analysis tool Understand J4). 

The approach and the tool are applied to the verification of safety properties of the Trampoline op- 
erating system 13), which is an open source automotive operating system compliant with OSEK/VDX. 
Environment models are generated using the assertions identified from the kernel code, and the kernel 
code itself is reduced by including only those extracted functions and their relevant code. The envi- 
ronment model is used to model-check/test the abstract code using CBMC and random testing. We 
compare their fault-detection capability, their comprehensiveness in terms of code coverage, and their 
efficiency in terms of resource consumption. 

The remainder of this paper is organized as follows. Section [2] briefly discusses related work and 
Section [3] provides the motivation for our work. Section [4] provides an overview of our approach and 
Section [5] presents the methods and the process for the automated environment generation technique. 
Section [6] explains the environment settings for the collaborative verification, followed by experimental 
results and the evaluation using Trampoline OS as a case example in Section|7] We conclude in Section[8j 

2 Related Work 

Environment modeling for efficient model checking has been an active research issue (221 [23l [T21 [23 
[l7l[20l. Reference |[20l is one of the earliest works concerning environment assumptions in verification. 
It introduced the observer concept to represent assumptions about the environment. The approaches for 
assumption generation were developed further in H2|23l[T3l|T9]|. Reference |[23l automatically generates 
the environment of Java programs from the specifications written by a user. HUE) are concerned about 
automatic partitioning, learning, or minimizing assumptions for compositional verification. None of 
them considers environment generation for both model checking and testing. 

Several specification-based environment generation methods exist: EH uses ADL to define pro- 
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tocols of Java components and constructs an environment for the ADL specification, [llj describes 
environmental assumptions in LTL and uses them to filter a universal environment, which is adopted in 
our approach to constrain the non-deterministic initial task model. Reference lf25l automatically gen- 
erates scripts in Promela from environment models for OSEK/VDX-based operating systems that are 
modeled in Uml diagrams. Their approach, however, models all basic objects in OSEK/VDX using 
UML class diagrams and state diagrams, from which all combinations of deterministic environments 
are generated and verified individually. The models are then used to automatically generate exhaustive 
test cases for the conformance testing of OSEK/VDX-compliant operating systems (8J. Their approach 
assures the exhaustiveness of test cases, but the scalability issue remains, as the number of test cases may 
increase exponentially. 

Program slicing ll24l has been a popular technique for reducing verification complexity for both 
model checking and testing. References Q and lTT4ll use slicing algorithms to explicitly detect def — 
use associations that are affected by a program change for efficient regression testing. Reference Q 
performs program slicing for C programs with respect to the alarms generated from value analysis. 
[|T5l integrates aggressive program slicing and a proof-based abstraction-refinement strategy for wireless 
cognitive radio systems. It is a representative example of using program slicing and bounded model 
checking for embedded software, but the slicing is integrated into the model checking process, and is 
thus not suitable for application in testing. 

3 Background 

3.1 OSEK/VDX 

Osek/vdx is a joint project of the automotive industry, which aims at establishing an industry standard 
for an open-ended architecture for distributed control units in vehicles [lj. The aim of Osek/vdx is 
to provide standard interfaces independent of application, hardware, and network, and ultimately, to 
save the development costs for non-application related aspects of control software. It is specialized for 
automotive control systems, removing all undesired complexities such as dynamic memory allocation, 
circular waiting for resources, multi-threading, and so on. Since its target system is safety-critical, it 
strictly prohibits uncontrolled dynamic behavior of the system. 

Conformation testing is a standard verification method for the certification of OSEK/VDX-based 
operating systems. However, conformation testing suites are typically insufficient to identify safety 
problems. As OSEK/VDX explicitly specifies more than 26 basic APIs, thorough conformation testing 
would require at least 26 x 2 x 3 test cases even if we assume two arguments per API and only boundary 
values for the arguments are chosen. The possible number of execution sequences for these 26 x 2 x 3 
test cases would rise to 156 factorials, a large number to be tested in practice. 

3.2 Trampoline 

Trampoline 121 is an open source, real-time operating system compliant with OSEK/VDX version 2.2.3. 
It is developed in ANSI C and can be ported to various hardware platforms such as Arm, POSIX, PPC, 
AVR, HCS12, CI 66, etc. Since it also supports POSIX, it can be test-run on a UNIX/Linux environment 
before being ported to an actual operational environment. As its target platform varies, its platform- 
dependent part is clearly structured in a separate module that combines with the kernel module at com- 
pile time. Access to the hardware- specific part is abstracted using extern variables and macros so that 
the main control logic does not need to be aware of the specific hardware feature. As illustrated in 
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Figure 1: Components of Trampoline 



Figure [T] development of an automotive software using Trampoline requires four components; (1) ap- 
plication source code, (2) kernel configuration generated from configuration description written in OIL 
(OSEK Implementation Language) using the Goil compiler, (3) generic OS kernel code compliant with 
the OSEK/VDX standard, and (4) platform-dependent kernel code. The generic OS kernel code imple- 
ments services for task management, resource management, interrupt handling, and event/counter/alarm 
management, providing corresponding APIs. 

3.3 Model checking using CBMC 

Formal verification methods based on model checking ifTUl are an effective technique for identifying sub- 
tle issues in software safety which is particulary important for embedded systems. Current technological 
advances in model checking enable engineers to directly apply the technique to program source code, re- 
moving the manual model construction process. CBMC |9 | is one of these model checking tools, which 
is capable of verifying almost full ANSI C. It can be used to verify buffer overflows, pointer safety, ex- 
ceptions and user-specified assertions. Furthermore, it can check ANSI C and C++ for consistency with 
other languages, such as Verilog. The main advantage is that it is completely automated and generates 
counterexample traces when a property in question is refuted. 

As with any other model checking tool, CBMC also suffers from the problem of scalability. When 
applied to the Trampoline kernel as a whole with an arbitrary sequence of API calls, for example, it ran 
out of memory for checking one assertion on a PC with 3GB of memory. 

4 Overall Approach 

Comprehensive verification, required by functional safety analysis, is too costly to be applied in practice. 
Reducing the cost while maintaining comprehensiveness is a challenging, but crucial task. Our approach 
attempts to achieve this goal with the following three strategies: 

1. Property-based environment generation: An environment of the operating system kernel is auto- 
matically generated using static code analysis for a given safety property. 
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Figure 2: Collaborative verification approach 

2. Property-based abstraction: The operating system kernel is abstracted by extracting only the code 
relevant to a given property. 

3. Collaborative verification using model checking and testing: Both model checking and testing are 
used complementarily for the verification of the abstract kernel code under the generated environ- 
ment model. 

Cost reduction is achieved through property-based environment generation and code abstraction. 
The efficiency of verification is increased by taking advantage of both verification techniques. Figure [2] 
is an overview of the suggested collaborative verification approach. Our approach uses both model 
checking and testing to complimentarily utilize their different capabilities when only limited resources 
are available. 

5 Environment Generation 

A straightforward way to include all possible task interactions with the operating system is to model 
the task with strongly connected states, where each state represents an API call to the kernel and each 
transition between states is not guarded. However, this includes too many spurious and/or impossible 
behaviors and increases the cost for verification as well as counterexample analysis; if 26 APIs are 
provided by the operating system, the task model would have at least 26 strongly connected states. Our 
approach tries to minimize unnecessary verification cost by using property -based extraction of dependent 
functions. 

5.1 Abstraction through static code analysis 

Given a property, we first extract the variables specified in the property, which is called Verification Tar- 
get Variables, and identify all the variables that are used to define the Verification Target Variable, called 
Extended Verification Target Variable. Then, functions modifying those Extended Verification Target 
Variables, called End _Level -Functions, are extracted. The prototypes of the End -Level -Functions are 



Counter Example 
Analysis 




74 



Property-based Code Slicing 




Figure 3: Backward and forward reachability analysis for environment generation 

used to construct an end-level environment model. The corresponding end-level abstract kernel code 
consists of all the End LeveLF unctions and their dependent code. The Root Level functions are identi- 
fied by performing backward reachability analysis from each End Lev el L unction. The prototypes of the 
Root Level functions are used to construct a root-level environment model. Its corresponding abstract 
kernel code is identified by performing forward reachability analysis from each Root Level function. 
The result of forward reachability analysis is also used to identify constraints for the end-level environ- 
ment model. 

Definition 1 Property -related variables: 

1. A Verification Target Variable is a variable that appears in the property specification. 

2. An Extended Verification Target Variable is a variable that a Verification Target Variable depends 
on. 

Definition 2 Classification of functions: 

1. An End Lev el L unction is a function that directly modifies, sets, or uses an Extended Verification 
Target Variable. 

2. A Root Level -Function is an API that is a terminal node of the called-by graph of an 
EndLevelLunction. 

From this process, we extract two types of functions for constructing different levels of environment 
models: (1) functions for root-level environments, and (2) functions for end-level environments. Figure[3] 
shows the conceptual diagram for the whole process. 

For a simple example, if a property in question is 



Property \ : assert(tpLfifo_rw[tplJi_prio].size>0), 
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then we first identify Extended Verification Target Variables and End JLev el functions for tpLfifo_rw 
and tpLh-prio. The identified set of End .Level-Functions for the variable tplJi.prio is { tpLget.proc, 
tpLput. preempt ed.proc, tpLput_new-proc, t pi ^schedule .fromjrunning } in the Trampoline kernel. 
An end-level environment model is constructed as non-deterministic calls to those end-level functions 
and its corresponding abstract kernel encompasses all the identified End Level functions and their de- 
pendent code. We then identify Root level functions for each of the End Level -Functions', For ex- 
amples, {ReleaseResource, Schedule, ActivateTask, SetEvent, TerminateTask, ChainTask, WaitEvent, 
Start OS} are Root Level -Functions for tpLget-proc, which are identified from its called-by graph. A 
root-level environment model consists of non-deterministic calls to those API functions and its corre- 
sponding abstract kernel encompasses all the identified root level functions and their dependent code. 



5.2 Implementation 
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Figure 4: Environment Generation 



The suggested approach was implemented and fully automated. Figure [4] shows the overall struc- 
ture of the automation. The source code of the Trampoline OS is analyzed by the static analysis tool 
Understand E), which creates a data repository of the analysis results from which information on vari- 
able/function dependencies can be extracted using a C plug-in. The environment generator first extracts 
target variables from the properties and then extracts Extended Target Variables and End-Level Func- 
tions by analyzing dependency relations among variables and functions. Root-Level Functions are then 
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extracted from the called-by graph for each End-Level Function. A Root-Level Function Scenario is 
generated as an arbitrary sequence of function calls of Root-Level Functions, which complies with the 
external constraints from the OSEK/VDX standard. Finally, the environment model is generated as an ar- 
bitrary sequence of End-Level Functions that complies with both the external and the internal constraints. 
The internal constraints are partial-order relations among End-Level Functions, which are generated from 
each Root-Level Function. 

The last step is the property-based abstraction of the original code. Since the environment generation 
step identifies all necessary End-Level Functions modifying the Verification Target Variables together 
with the ordering relation among them in call sequences, verification requires only the source code 
of those End-Level Functions plus codes for preprocessing them. Therefore, for each safety property, 
verification is performed using the environment model generated with the tool and the property-based 
abstract code. 



6 Setting up Environments for Collaborative Verification 




Figure 5: Scenario generation process for random testing 

CBMC and testing require different settings for their verification environments even with the same set 
of End .Lev elJ? unctions', CBMC requires only the algorithm of non-deterministic function calls. Ran- 
dom testing (both root-level and end-level), however, requires explicit function call sequences generated 
from a given environment model. Figure|5]shows the process for generating such function call sequences, 
which can be applied to both End-Level and Root-Level environment models. It repeats the selection and 
checking process by arbitrarily selecting a function and checking whether the selected function satisfied 
the constraints or not. We have implemented an OSEK/VDX simulator to check the external constraints 
in the process of root-level sequence generation. Internal constraints are identified by call graph analysis. 
Details are described in the following two sub-sections. 

We note that the checking the constraints is not necessary to ensure the correctness of the scenario 
generation, but it is essential for making the verification efficient, since otherwise the verification pro- 
duces too many false errors. 
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6.1 Root-Level Scenario Generation 



Root-Level Test Environment 




Figure 6: Root-Level scenario and the testing environment 

Root-Level scenario generation is based on the scenario generation process illustrated in Figure [5] A 
Root-Level Function Scenario is an arbitrary sequence of function calls of Root-Level Functions, and it 
complies with the external constraints from the OSEK/VDX standard. Figure [6] shows the Root-Level 
test environment for the Trampoline OS. Since an OIL file is required for testing, an OIL file is specified 
as an input, which is compiled with the extracted kernel code and the random sequence of Root-Level 
Functions generated from our Root-Level scenario generator. 

There are two things to be done before implementing a scenario generator. First, every constraint 
specified in the OSEK/VDX standard should be identified. Second, an OSEK/VDX Simulator - an 
abstract OSEK/VDX model - should be implemented in order to trace all the changes and fully observe 
constraints. 

6.1.1 Identification of external constraints 

The OSEK/VDX standard explicitly specifies constraints among the APIs. The description column of 
Figure [7] lists some of the constraints manually identified from the standard. These constraints are rep- 
resented as pre-conditions with respect to other APIs. For example, the API function TerminateTask 
can be called only if the task has been activated either by ActivateTask or ChainTask. Therefore, we set 
{ActivateTask, ChainTask} as preconditions of TerminateTask. Figure [7] shows a couple of precondi- 
tions of other API functions. 

Identified constraints are then imposed in the Root-Level scenario generation process illustrated in 
Figure [5] Figure [8]is an example algorithm of the constraint checking. 

6.1.2 OSEK/VDX Simulator 

To fully consider all the identified constraints, it is necessary to trace changes that previous function calls 
have made. For example, if ActivateTask( tl ) is chosen as the first Root-Level Function in a scenario, task 
tl should be marked as READY task, for further scenario validation. This process is fully automated by 
implementing an OSEK/VDX simulator. 

The OSEK/VDX simulator traces run-time information such as list of resources, list of events, list of 
task models, reference to running task, ready queue (priority queue), and waiting queue. It provides Root- 
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Figure 7: Constrant list extrected from OSEK/VDX spec 



bool canReleaseResource( Resource rl ){ 
if ( running task doesn't exist ) 
return false; 

else if ( running task is not holding resource rl ) 
return false; 

else // when running task is holding the resource rl 
return true; 

} 



Figure 8: ReleaseResource constraint checker 

Level Function calls just like OSEK/VDX APIs. When one of these procedures is called, it simulates 
the behavior of OSEK/VDX. The simulator includes task model, task scheduler, event management, and 
resource management. Figure [9] shows the overall process of the OSEK/VDX simulator. 

A randomly chosen function is added to the body of the running task by a module called Scenario 
Generator. When there is a function in the body of the running task that has not been executed by the 
simulator, it executes the function. If scheduling is necessary, it calls the scheduler and executes all the 
functions that already exist in the body of the preempted task, when preemption occurs. This process is 
repeated until there is no function left to execute, and the Scenario Generator is requested to generate 
another function randomly. 

Whether scenario generation has been completed or not can be determined by checking all tasks 
that are initially generated from the given OIL file. If every task has completed its execution with 
TerminateTask or ChainTask, it can be determined that the scenario generation is completed. 

6.2 End-Level Environment Model 

The End-Level scenario generation process is also based on the scenario generation process illustrated 
in Figure [5] It executes an arbitrary sequence of End Level functions, which is chosen from among 
the list of End -LeveLFunctions serving as an environment of CBMC model checking and End-Level 
random testing. 

This process considers internal constraints among End-Level functions; since End-Level functions 
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Figure 9: OSEK/VDX simulator 



can only be called by API-Level functions, the pre-conditional constraints of Root-Level functions iden- 
tified from the OSEK/VDX constraints must be implicitly obeyed by the End-Level functions. These 
implicit constraints can be identified by analyzing call-graphs of Root-Level functions and their pre- 
conditional relations. We call such implicit constraints internal constraints. 
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Figure 10: End-Level Functions called by each API-Level Function 

For example, if we consider the Root-Level APIs and their corresponding End-Level functions, the 
external constraint Wait Event can be called after SetEvent is called can be re-interpreted as tpl.get.proc 
can be called after tpLputJiew-proc and t pi schedule _f rom .running are called, which can be writ- 
ten in a regular expression (tpLput_new-proc t pi schedule -fromjrunning {End -Level -Function — 
tpl-get-proc)* tpLget-proc)* . 

Checking constraints of this kind cannot be done using the OSEK/VDX simulator because the End- 
Level functions include implementation- specific function names that cannot be modeled from the stan- 
dard. Instead, the internal constraints are simplified using the characteristic that a function cannot 
be called more times than its preceding functions in the partial order relation. The example inter- 
nal constraint can be simplified as The number of tpl_get_proc calls cannot exceed the one of either 
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tpl-putJiew-proc or t pi schedule _f rom_running. The constraint checker keeps track of the number of 
each End-Level functions calls and checks (#tpLget.proc < #tpLput_new-proc) && (#tpLget_proc < 
# t pi schedule -from .running). 



7 Experiments 

We have conducted a series of experiments to show the impact of our approach using CBMC model 
checking, End-Level random testing, and Root-Level random testing. The target verification properties 
are three functional safety properties from the Trampoline kernel: 



assert {tplJi-prio ^ — 1) (1) 
assert (tplJcern ^ NULL) (2) 
assert (t pi. kern -> state RUNNING) (3) 

tplJi-prio is the value of the highest-priority task in the ready queue in the Trampoline kernel. 
tpLh.prio ^ — 1 is supposed to be true whenever rescheduling is necessary. tplJkern stores the key in- 
formation of the currently running task. tplJcern ^ NULL and tplJcern — >> state == RUNNING checks 
if the state of the running task is RUNNING when the scheduler is called. 

We performed verification of these three assertions using the model checker CBMC, Root-Level ran- 
dom testing, and End-Level random testing. The verification cost in terms of the number of verification 
conditions and the resource requirements was measured for CBMC verification. Branch coverage was 
measured using the Squish Coco code coverage tool El for random testing. All experiments were per- 
formed on Linux Fedora 16 OS, with Intel Xeon 3.4GHz e3-1270 processor and 32GB of 1333MHz 
DDR3 RAM. 
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Figure 1 1 : Time and memory space to verify with CBMC 



Figure 1 1 shows the time and memory space it took to verify the Trampoline operating system with 
the End-Level Environment Model using the model checker CBMC. Time and memory space increase 
exponentially as the length of the End-Level function calls (unwind value) increases. CBMC verifies the 
assertions by searching through every possible scenario with the length of the unwind value, making it 
a powerful method. With the given resources, CBMC reported no counter examples up to the unwind 
value 10, but was not able to finish its verification process for the unwind value 15 after 6 days. We 
do not report the result of CBMC model checking using Root-Level environment models since it is too 
costly to perform even with the value of unwind option 10. 
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Figure 12: Coverage, time, and memory space to verify with Root-Level Random Testing 
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tpl_h_prio != -1 

code size : 437 lines 

3 End -Level Functions / 19 Functions in total 


tpLkern != NULL 

tpl_kern-> state == RUNNING 
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7 End-Level Functions / 32 Functions in total 
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Figure 13: Coverage, time, and memory space to verify with End-Level Random Testing 



There are a few dozens of extracted functions in random testing environments, including End-Level 
Functions, as illustrated in Figures [12] and [13] Due to a lack of space, only the test results of End-Level 
Functions are described. Root-Level Random Testing (Figure [T2| is much faster (less than 1/100 sec- 
onds), consumes little memory (up to 2.64MB of memory), and achieves a certain level of test coverage 
quickly, but the coverage does not improve after test sequences of length 34. In End-Level Random Test- 
ing (Figure [T3]), a test sequence of length 100 achieves a certain level of coverage both for tplJi_prio and 
t pL kern. The coverage stays the same afterwards. End-Level Random Testing required around 1.06 1.07 
MBytes of memory. 

For many cases, the coverage did not increase even with lengthier test cases. There can be two reasons 
why some part of the code are unreachable. The first reason is exception handling; parts of the code for 
exception handling are never reached unless an exceptional situation occurs. The second reason is that 
some variables in conditional statements are not included in the Extended Verification Target Variable. So 
the behavior of updating these variables might not be fully extracted, which can make some conditional 
statements fixed. An example of this case is illustrated in Figure [14] This conditional statement is only 
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void tpl_schedule_f rom_running (void) { 



// READ Y_ AN D_N E W 

if ( tpl kern.running->state == READY AND NEW ) 
{ 

tpl_init_j?roc ( tpl_kern . running_id) ; 

} 



} 



Figure 14: Example of uncovered conditional statement 



tpl_get_proc 


-> tpl_h_prio : 


2, 


taskNum 


4, 


activationCount(T1, T2, T3) 
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tpl_put_new_proc 


-> tpl_h_prio : 
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255, 1 
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tpl_get_proc 


-> tpl_h_prio : 
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0, 


tpl_schedule_from_running 


-> tpl_h_prio : 


-1, 


taskNum 


3, 


activationCount(T1, T2, T3) 


0, 


0, 



RandomTest: RandomTest .c:505: tpl_schedule_from_running: Assertion x tpl_h_prio ! = -V failed. 



Figure 15: Error caused by overflow 



executed when tpl schedule -from -dying or tpLactivateJask is executed before this code. But these two 
functions are out of boundary in this model because the model is generated only with regard to the 
property tpLh_prio != 0. Thus coverage cannot be increased, and testing terminates. 

In terms of comprehensiveness, CBMC is the most powerful method, verifying every possible sce- 
nario within the same length, but it is limited by the length. As shown in Figure [TT] the verification 
cost increases exponentially as unwinding depth increases. Therefore, CBMC cannot detect potential 
faults that can be identified only in long task scenarios. Unlike verification with CBMC, the length of 
the scenarios is not limited in random testing since the cost is much cheaper than CBMC, as shown in 
Figure [T3| Though it cannot be comprehensive, it can be more effective in stress testing, since the length 
of the test sequences can be sufficiently long. 

End-Level Random Testing did in fact, catch some overflow errors in the Trampoline kernel as illus- 
trated in Figure [15] These errors have occurred because the size of the variables saving the activation 
count is limited to 8 bits; the second line of Figure [H] shows that Task! has the activation count 255, but 
adding another activation changed its value to 0. So Trampoline has changed the value of tplJi-prio to 
-1, meaning that the process table has no activated task available. The variable size is implementation- 
specific and is not constrained to 8 bits, neither in the OIL specification nor in the OSEK/VDX specifi- 
cation. This problem could be addressed by constraining the size to 8 bits in the OIL specification. 

A Model checker could find this type of potential faults if we can set the value of the unwind option 
larger than 255, but our experiments could not identify it due to resource limitations. End-Level Random 
Testing is appropriate for finding this kind of errors because the cost does not increase much even with 
lengthy test scenarios. 

We could not identify the same fault using Root-Level Random Testing, either. The main reason 
is that Root-Level Random Testing is coupled with a pre-defined OIL configuration file. The OIL file 
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specifies the typical system configuration, and thus, activating a task over 255 times is not likely to 
happen unless we specifically aim at stress testing. End-Level Random Testing is more effective in stress 
testing, since it is not constrained by the system configuration and can test abnormal cases. 

API-Level Random Testing, however, is beneficial in that it is not necessary to do additional API- 
Level analysis when testing identifies faults, which is necessary in model checking and End-Level Ran- 
dom Testing. 

8 Conclusion 

This paper presented methods and tools for environment generation and code abstraction to improve the 
efficiency of verification using model checking and testing. The effect of using the suggested approach 
was demonstrated through a series of experiments using the Trampoline operating system as a case 
example. The benefit of property -based environment generation is two-fold: (1) it reduces verification 
cost by reducing the target code and by limiting its environment to the task interaction scenario relevant 
to the verification property, and (2) it simplifies the analysis process and localizes the verification activity 
by focusing on the points of interest. 

The experiments revealed relative pros and cons of the three verification methods and identified 
potential safety faults, which suggests the following collaborative use of model checking and testing; 

1. Apply End-Level Random Testing first for stress testing. 

2. Apply Root-Level Random Testing to conform the errors identified through End-Level Random 
Testing. 

3. Apply model checking using CBMC last for comprehensive verification within a limited scope. 

Our tool still needs some improvements. First, conditional dependencies need to be considered so 
that test coverage can be improved. Second, Root-Level scenario generation currently assumes a fixed 
OIL configuration. We would like to relax the condition so that an arbitrary OIL can be handled by the 
tool. 
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